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[An Improved Method for 
Displaying Temporal Changes in 
Spatially Matched Images 

Background of Invention 

[0001] The present invention relates generally to imaging systems and more particularly 
to a method to Improve the display of temporal changes. Imaging devices, such as x- 
ray machines, are widely used in both medical and industrial applications. Imaging 
devices often use temporal processing to track change in an object over time. 

[0002] Temporal processing systems typically include the following general modules: 

acquisition storage module, segmentation module, registration module, comparison 
module, and reporting module. The input images are 1 -D. 2-D, 3-D, derived, 
synthesized, or montaged, where multiple separate images from a single time point 
are combined to provide a larger composite, seamless image. Additionally, 1 -D 
images can be montaged to produce 2-D and 3-D images, for example a CT scout- 
view. 

[0003] The detection of change in medical Images of a patient acquired at two different 
instances in time has great potential for improving diagnosis. The advent of digital 
imaging allows computer-assisted detection and identification of these changes and 
the creation of a "dissimilarity image" containing the change Information. This 
dissimilarity Image can be read by a human reader or can become the input to an 
automated analysis device such as a CAD (computer assisted diagnosis) algorithm. 

[0004] 

Currently, as part of Mitsubishi Space Software"s "temporal 
subtraction"application, dissimilarity images are calculated using a simple pixel-by- 
pixel subtraction of registered images. Simple subtraction, however, results in images 
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with poor contrast, and is not substantially robust when the two initial images are 
acquired using different techniques. Simple subtraction also does not incorporate an 
indication of the confidence in the magnitude of the dissimilarity measurement. 

[0005] For a temporal subtraction image, the resulting pixel values (and hence the 

displayed gray-levels) are proportional to the difference or dissimilarity in pixel value 
between two input images acquired with temporal separation. 

[0006] Input images are often registered and processed to compensate for several factors 
such as: the difference in positioning of the subject during the two image 
the difference In acquisition parameters, the difference in the bit resolution of the 
images, and the differences in any pre or post processing that may have been applied 
to the images. 

[0007] Any errors in registration of the two images may result in significantly large values 
in the dissimilarity image. For example, if the resulting registration is not perfect, the 
temporal analysis image of a subject that is identical at both imaging times, is not 
uniform whereas the desired result is an image with no contrast. Non-zero elements 
appear at the positions where the registration was not exact. These non-zero 
elements represent artifacts that could be mistaken for a temporal change in the 
subject. 

[0008] Even an ideal registration algorithm produces artifacts, if for no other reason than 
noise in the image acquisition process. With a poorly chosen display mapping, 
differences In the noise texture between the two temporally separated images are 
visible and potentially distracting to the observer. In the case of imperfect registration, 
it is possible that errors in registration will be displayed with higher contrast than 
physical changes in the patient. 

[0009] The disadvantages associated with current, imaging systems have made it 

apparent that a new technique for temporal processing and display is needed. The 
new technique should substantially increase accuracy of information acquired 
obtained from temporal processing. The present invention is directed to this end. 

Summary of Invention 
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[001 0] In accordance with one aspect of the present invention, a temporal image 

processing system includes a temporal processing controller adapted to receive a first 
image signal and a second image signal from an image controller. The temporal 
processing controller includes a registration module adapted to register a region of 
interest of the first image signal and the second image signal and generate therefrom 
a registration signal. The temporal processing controller further includes a confidence 
module adapted to receive the registration signal and further adapted to determine a 
confidence map therefrom. The confidence map is adapted to enhance a contrast of a 
temporal change in the object relative to a contrast due to at least one 
misregistration. 

[001 1] In accordance with another aspect of the present invention, a temporal image 
processing method includes scanning an object and generating a first image signal 
and a second image signal therefrom. The method further includes receiving the first 
image signal and the second image signal in a registration module and registering at 
least one region of interest of the first image signal and the second image signal such 
that at least one cost signal is generated. A confidence map is generated from the 
cost signal thereby enhancing a temporal change contrast between the first image 
signal and the second image signal. At least one distinction is generated between a 
contrast of temporal changes between the first image signal and the second image 
signal and a contrast resultant from at least one misregistration. 

[001 2] Additional advantages and features of the present invention will become apparent 
from the description that follows and may be realized by the instrumentalities and 
combinations particularly pointed out in the appended claims, taken in conjunction 
with the accompanying drawings. 

Brief Description of Drawings 

[001 3] ^ ^^^^ complete understanding of the invention, there will now be described 

some embodiments thereof, given byway of example, reference being made to the 
accompanying drawings, in whichiFIGURE 1 is a diagram of an imaging system in 
accordance with a preferred embodiment of the present invention:FIGURE 2 is a 
schematic diagram of FIGURE 1 ; FIGURE 3 is a schematic diagram of the temporal 
processing unit of FIGURE 1 ; andFIGURE 4 is a block diagram of a temporal image 
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processing method, in accordance with a preferred embodiment of the present 
invention. 

Detailed Description 

[0014] The present invention is illustrated with respect to a temporal image processing 
system 10, particularly suited to the medical field, specifically to chest x-ray imaging. 
The present invention is, however, applicable to various other uses that may require 
temporal imaging, such as non-destructive testing and other industrial applications, 
as will be understood by one skilled in the art. 

[001 5] Referring to FIGURES 1 and 2, an image controller 1 2, including a temporal 
processing controller 14, coupled to a scanning unit 1 6, in accordance with a 
preferred embodiment of the present invention, Is illustrated. The scanning unit 
16includes, for example, an x-ray source 1 8 coupled to a gantry 20. generating an x- 
ray flux 22, which passes through an object 24 (e.g. a patiept) on a table 25. The 
system 10 further includes an x-ray detector 26, also coupled to the gantry 20. 

[001 6] The image controller 1 2, including the temporal processing controller 1 4 and 
various other widely known imaging control and display components, receives the 
detector signals and responds by generating a first and a second image signal. The 
imaging controller 1 2 also includes, for example, an operator console 28, a monitor 
and user interface 29 an x-ray controller 30, a table control 32. a gantry motor 
control 34, a mass storage 38, an image reconstructor 41 and a data acquisition 
system (DAS) 42. all of which couple to a host computer and display 43. are well 
known in the art, and will be discussed later. 

[001 7] Referring to FIGURE 3, a schematic diagram of the temporal processing controller 
1 4 of FIGURE 1 is illustrated. One embodiment of the temporal imaging controller 1 4 
includes an acquisition storage module 40, a processing module 42, a segmentation 
module 44, a registration module 46, a confidence module 48, a comparison module 
50, and a report module 52. The aforementioned modules are either software 
algorithms or discrete circuit components or a combination thereof, as will be 
understood by one skilled in the art. 

[001 8] Referring to FIGURES 1 , 2 and 3, the acquisition storage module 40 contains the 
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acquired images. For temporal change analysis, a data retrieval system retrieves the 
data from storage (e.g. the mass storage unit 38) corresponding to an earlier time 
point or from the image reconstructor 41 , which receives signals from the data 
acquisition system (DAS)42. 

[001 9] As an illustrative example, only two images I ^ and I ^ corresponding to two time 

points t and t are Included, however one skilled in the art will realize that the 
^ 1 2 

general approach can be extended for any number of images in the acquisition and 
temporal sequence. The temporal comparison image is denoted ' ^ _2 * ' ] ' 2 
are either the original unprocessed images from an acquisition system or alternately, 
post-processed versions of the original images. One skilled in the art will realize that 
the acquisition storage module 40 can include images from almost any digital imaging 
source including digitized versions of analog, film images. 

[0020] The processing module 42 receives the two images, I ^ and I ^ , from the 

acquisition storage module 40 and "normalizes"them to account for differences in 
acquisition techniques or differences in pre- and post-processing methods. To 
illustrate, in x-ray images, If the first image I ^ has half the exposure of the second 
image I ^ . the gray levels in the first image are multiplied by two before any further 
comparison to the second image. This multiplication adjusts for the differences in 
overall image intensity due to, for example, differences in x-ray dose. 

[0021] The segmentation module 44 receives the normalized images from the processing 
module 42 and. through automated or manual operation in the user interface 29 or 
operator console 28, isolates regions of interest of I ^ 54 and I ^ 56. Often is the case 
that the entire image is the region of interest. 

[0022] The registration module 46 receives the region of interest signals 54, 56 from the 
segmentation module 44 and provides methods of registration and therefrom 
generates a registration signal. If the regions of interest 54, 56 for temporal change 
analysis are small, rigid body registration transformations including translation, 
rotation, magnification, and shearing are sufficient to register a pair of images from t 
^ and t 2 . If, however, the regions of interest 54, 56 are large including almost the 
entire image, warped, elastic transformations are applied. 
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[0023] One way to implement the warped registration is to use a multi-scale, multi- 
region, pyramidal approach. For this approach, a different cost function highlighting 
changes is optimized at every scale. Such cost functions are correlation methods, such 
as mathematical correlation and sign-change measurement, or statistical methods 
such as entropy measurements and mutual information. 

[0024] For warped registration, images are re-sampled at a given scale and subsequently 
divided Into multiple regions. Separate shift vectors are calculated for different 
regions. Shift vectors are interpolated to produce a smooth shift transformation, 
which is applied to warp one of the images. The images I ^ , I ^ are re-sampled and 
the warped registration process repeats at the next higher scale until the pre- 
determined final scale is reached. In other situations, a combination of rigid 
registration and elastic transformations is used. 

[0025] The confidence module 48 receives the cost function and a figure of merit of the 
cost function and uses them to determine a confidence map. The resulting map 
enhances the contrast of temporal changes in the object relative to the contrast due to 
misregistrations. This highlights anatomical changes relative to minor errors in 
registration. It is important to note that the cost function provides extreme values 
irrespective of whether the non-zero pixel values in the dissimilarity image are due to 
inadequacy of the registration methods or due to true anatomical changes. 

[0026] An alternate embodiment includes separating out whether the change was due to 
anatomical change or due to inaccuracies in registration. This is accomplished in 
several different methods. All these methods make use of the prior knowledge of the 
object 24, the anticipated modes of motion, or the anticipated disease-state. 

[0027] For example, the bones in the object 24 tend to move as rigid bodies, objects 

within an organ or tissue structure tend to move together in a smooth and continuous 
manner. Knowledge of the characteristic shape, expected extent, etc of a disease state 
is alternately used. For example, for detecting nodules, circularity metric in 2D or 
sphericity metric in 3-D is used with anticipated limits on these metrics. 

[0028] 

The comparison module 50 receives the I ^ region of interest signal 54 and the 
registration signal, computes a dissimilarity measure between the registered images. 
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and therefrom generates a comparison signal. 

[0029] Registered image comparison is performed in several ways. In addition to, or 
instead of. using a simple subtraction between the registered images to get the 
dissimilarity image, an enhanced division method is used, which is described as (LI * 
l_2)/(L2 * L2 + 0 ). 

[0030] The confidence image applies to at least two embodiments. In the first 

embodiment, the confidence map is integral to the creation of a dissimilarity image. 
For example, in one embodiment of the present invention. Phi ( <t> ) is spatially 
dependant upon the confidence map. To provide additional flexibility, a sensitivity 
metric determines the relative strength of the confidence weighting. This is illustrated 
by the dotted line connecting the confidence module and the comparison module. The 
user control or other known control device adjusts the relative importance thereof. 
The control Is embedded within the application or adapted for user operation. 

[0031] For example, by extreme adjustment of the control, a user calculates a pure 
dissimilarity image (all changes are equally weighted), or by the opposite extreme 
adjustment of the control the user calculates a scaled dissimilarity image in which the 
pixel values are scaled relative to the confidence that they represent anatomical 
changes and not registration errors. 

[0032] In the second embodiment, the confidence map is used only to weigh the display 
such that the image resulting from the enhanced division combines with the 
previously derived confidence information, as discussed below in regards to the report 
module 52. 

[0033] 

The report module 52 receives the comparison signal and provides the display and 
quantification capabilities for the user to visualize and quantify the results of temporal 
comparison. Results of temporal comparisons are simultaneously displayed on a 
second display unit with either image I ^ or I ^ . Or a superposition of I ^ onto I ^ or 
I ^ generates with a logical operator based on pre-specified criterion, as will be 
understood by one skilled in the art. For quantitative comparison, color look-up tables 
for the overlaid images are used as an illustrative example. Alternate quantitative 
comparison techniques include numerical classification, segmentation of the different 
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image sections, or other known discrete identification techniques. The resulting 
combination is realized with a multi-color overlay display. 



[0034] 



Another way to combine change and confidence information Is to modify locally 
the dissimilarity Image, based on the confidence map. Such modifications affect, 
locally, the colors used to display the difference image or the gray level contrast Itself. 



[0035] 



To provide additional flexibility, a sensitivity metric determines the relative 
strength of the confidence weighting. This is illustrated by the dotted line connecting 
the confidence module and the report module 58. The user control adjusts the 



sensitivity metric relative importance. 



[0036] 



For example, through extreme adjustment, the Monitor 29 displays a pure 



dissimilarity image (all changes are equally weighted), or through the opposite 
extreme adjustment of user control, the monitor 29 displays a scaled dissimilarity 
image in which the pixel values are scaled relative to the confidence that they 
represent anatomical changes and not registration errors. This display incorporates 
either color, numerical or other known methods for distinguishing characteristics to 
highlight regions of interest. 

[0037] In addition to observer display, the resultant images are further analyzed 

quantitatively. For example the use of other statistical measures, such as entropy, or 
pattern recognition techniques including CAD perform further automated analysis of 
the results. 

[0038] The present invention is illustrated with respect to x-ray and computed 

tomography (CT) systems, however it is alternately used for any type of imaging 
system including magnetic resonance imaging (MRI), mammography, vascular x-ray 
imaging, bone scanning, PET, radionuclide imaging, ultrasound imaging, optical 
Imaging, etc. Further embodiments include non-medical applications, such as weld 
inspection and metal inspection. Essentially, anything application that uses an 
system to make 1 , 2 or 3 dimensional Images or 1 , 2, or 3 dimensional montaged 
Images is included. 

[0039] Typical scanning units include an x-ray source 1 8 coupled to a gantry 20. The x- 
ray source 1 8 generates an x-ray flux 22, which passes through a scanned object 26 

APP ID=10064548 Page 8 of 21 



on a table 25. An x-ray detector 26 is also typically coupled to the gantry 20 such that 
the detector 26 receives the x-ray flux 22. Detector specific corrections or calibrations 
are then engaged, as will be understood by one skilled in the art. 

[0040] The x-ray source 1 8 is embodied as a flat panel x-ray source or an extended x- 
ray source (e.g. Imatron), or a standard x-ray tube. The x-ray source 18 is activated 
by either the host computer 43 or an x-ray controller 30, as will be understood by one 
skilled in the art. The x-ray source 1 8 sends the x-ray flux 22 through the object 24 
on the moveable table 25 controlled by the table control device 32 acting in response 
to signals from the host computer 43, as will be understood by one skilled in the art. 

[0041] The embodied gantry 20 is a ring shaped platform that rotates around the 

scanned object 24 in response to signals from the gantry motor control 34, as will be 
understood by one skilled In the art. 

[0042] For a single image I the x-ray flux 22 from the x-ray source 1 8 passes through 

1 » 

the object 24 and impinges on the x-ray detector 26. The signal passes to the host 
computer and display 43. where the signal is converted to a gray level corresponding 
to the attenuation of an x-ray photon through the patient. The image is then stored in 
the mass storage unit 38 or received in the temporal imaging controller 14. 

[0043] The detector 26 is typically located opposite the x-ray source 1 8 to receive x-ray 
flux 22 generated therefrom, 

[0044] The host computer 43 receives detector signals. The host computer 43 also 

activates the x-ray source 18 through signals from the operator console 28 or user 
interface 29; however, alternate embodiments include independent activation means 
for the x-ray source 18. The present invention includes an operator console 28 for 
control by technicians, as will be understood by one skilled in the art. 

[0*045] One embodiment of the present invention incorporates use of temporal imaging 
for the scout scan on an imaging system 10. During a scout scan from the x-ray 
source 1 8 to the detector elements 26. the x-ray tube remains stationary while the 
patient table 25 translates under the x-ray flux 22. This results in a two-dimensional 
image ideal for qualitative information and for locating the desired position for 
scanning during further temporal imaging. 



APP ID=10064548 



Page 9 of 21 



[0046] Referring to FIGURE 4, a block diagram of a temporal image processing method 
100 is illustrated. Logic starts in operation block 102 where the object is scanned at 
different periods of time to generate image signals. 

[0047] Operation block 1 04 then activates, and the temporal processing controller 

acquires the image signals through the acquisition storage module, normalizes the 
images in the processing module, and isolates regions of interest in the segmentation 
module. Operation block 106 then activates, and the temporal processing controller 
registers the images. 

[0048] In operation block 1 08 the region or area of concern is analyzed. Minor regions 
are analyzed with at least one of translation criteria, rotation criteria, magnification 
criteria, or shearing criteria, thereby generating a cost signal (i.e. cost function or a 
figure of merit of the cost function) in operation block 1 1 4. 

[0049] Major areas are analyzed through at least one warped transformation criterion, 
thereby generating a cost signal (i.e. cost function or a figure of merit of the cost 
function). 

[0050] In operation block 1 1 4, the cost signal from a major region or a minor region is 
used to generate a confidence map in operation block 1 16. 

[0051] Operation block 1 1 8 activates, and at least one distinguishing characteristic 

between the contrast of temporal changes between the images and contrast due to 
misregistration is generated. 

[0052] A critical piece of temporal subtraction technology is the display of the results to 
the observer. It is critical that the observer understands the display processing chain. 
As such, even though the mapping of subtraction results to displayed gray levels Is 
likely to consist of embedded software, not accessible to the end-user, the specifics 
of the algorithm is disclosed so that the observer understands at what is viewed and 
can judge the clinical significance of the displayed gray levels using prior knowledge 
of the process and understanding of imaging physics, anatomy, and pathology. 

[0053] 

The additional inclusion of a user variable, or exam dependant sensitivity metric 
to determine the relative strength of the confidence weighting provides additional 
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flexibility. Since the user chooses to view anything from a pure dissimilarity image to 
dissimilarity image scaled relative to confidence in registration the user becomes 
familiar with the system and learn to judge the clinical significance of the displayed 
changes. 

[0054] In operation, the temporal image processing method includes scanning an object 
and generating a first image signal and a second image signal therefrom. A 
registration module receives the signals, and registers at least one region of interest 
of the first image signal and the second image signal such that at least one cost signal 
is generated. A confidence map is generated from the cost signal thereby enhancing a 
temporal change contrast between the first image signal and the second image signal. 
At least one distinguishing characteristic between a contrast of temporal changes 
between the first image signal and the second image signal and a contrast resultant 
from at least one misregistration is also generated. 

[0055] From the foregoing, it can be seen that there has been brought to the art a new 
temporal image processing system 10. It is to be understood that the preceding 
description of the preferred embodiment is merely illustrative of some of the many 
specific embodiments that represent applications of the principles of the present 
invention. Numerous and other arrangements would be evident to those skilled in the 
art without departing from the scope of the invention as defined by the following 
claims. 
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